<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]-->
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 1.5.8">
<title>Concourse Pipeline (Cloud Foundry)</title>
<link rel="stylesheet" href="css/spring.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
</head>
<body class="article toc2 toc-left">
<div id="header">
<div id="toc" class="toc2">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel1">
<li><a href="#concourse-pipeline-cf">Concourse Pipeline (Cloud Foundry)</a>
<ul class="sectlevel2">
<li><a href="#concourse-pipeline-step-by-step-cf">Step-by-step</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="concourse-pipeline-cf"><a class="link" href="#concourse-pipeline-cf">Concourse Pipeline (Cloud Foundry)</a></h2>
<div class="sectionbody">
<div class="admonitionblock important">
<table>
<tr>
<td class="icon">
<i class="fa icon-important" title="Important"></i>
</td>
<td class="content">
In this chapter, we assume that you deploy your application
to Cloud Foundry PaaS.
</td>
</tr>
</table>
</div>
<div id="concourse" class="paragraph">
<p>The Cloud Pipelines repository contains opinionated
Concourse pipeline definitions. Those jobs form an empty pipeline and an
opinionated sample pipeline one that you can use in your company.</p>
</div>
<div class="paragraph">
<p>There following projects take part in the  <code>microservice setup</code> for this demo.</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics">Github Analytics</a>: The app that has a REST endpoint and uses messaging&#8201;&#8212;&#8201;part of our business application.</p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-webhook">Github Webhook</a>: Project that emits messages that are used by Github Analytics&#8201;&#8212;&#8201;part of our business application.</p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-eureka">Eureka</a>: Simple Eureka Server. This is an infrastructure application.</p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics-stub-runner-boot">Github Analytics Stub Runner Boot</a>: Stub Runner Boot server to be used for tests with Github Analytics and uses Eureka and Messaging. This is an infrastructure application.</p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="concourse-pipeline-step-by-step-cf"><a class="link" href="#concourse-pipeline-step-by-step-cf">Step-by-step</a></h3>
<div class="paragraph">
<p>If you want only to run the demo as far as possible by using PCF Dev and Docker Compose, do the following:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><a href="#concourse-fork-cf">Fork repos</a></p>
</li>
<li>
<p><a href="#concourse-start-cf">Start Concourse and Artifactory</a></p>
</li>
<li>
<p><a href="#concourse-deploy-cf">Deploy infra to Artifactory</a></p>
</li>
<li>
<p><a href="#concourse-pcfdev-cf">Start PCF Dev (if you don&#8217;t want to use an existing one)</a></p>
</li>
<li>
<p><a href="#concourse-fly-cf">Setup the <code>fly</code> CLI</a></p>
</li>
<li>
<p><a href="#concourse-credentials-cf">Setup your <code>credentials.yml</code></a></p>
</li>
<li>
<p><a href="#concourse-build-cf">Build the Pipeline</a></p>
</li>
<li>
<p><a href="#concourse-run-cf">Run the <code>github-webhook</code> Pipeline</a></p>
</li>
</ol>
</div>
<div class="sect3">
<h4 id="concourse-fork-cf"><a class="link" href="#concourse-fork-cf">Fork Repositories</a></h4>
<div class="paragraph">
<p>Four applications compose the pipeline</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-webhook">Github Webhook</a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics/">Github Analytics</a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-eureka">Github Eureka</a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics-stub-runner-boot">Github Stub Runner Boot</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>You need to fork only the following repositories, because only then can you tag and push the tag to the repository:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-webhook">Github Webhook</a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics/">Github Analytics</a></p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="concourse-start-cf"><a class="link" href="#concourse-start-cf">Start Concourse and Artifactory</a></h4>
<div class="paragraph">
<p>You can run Concourse + Artifactory locally. To do so, run the
<code>start.sh</code> script from this repository. The following listing shows the script:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">git clone https://github.com/CloudPipelines/concourse
cd concourse/demo
./setup_docker_compose.sh
./start.sh 192.168.99.100</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>The <code>setup_docker_compose.sh</code> script should be run only once, to allow
generation of keys.</p>
</div>
<div class="paragraph">
<p>The <code>192.168.99.100</code> param is an example of an external URL of Concourse
(equal to the Docker-Machine IP in this example).</p>
</div>
<div class="paragraph">
<p>Then Concourse runs on port <code>8080</code>, and Artifactory runs on <code>8081</code>.</p>
</div>
<div class="sect4">
<h5 id="concourse-deploy-cf"><a class="link" href="#concourse-deploy-cf">Deploy the infra JARs to Artifactory</a></h5>
<div class="paragraph">
<p>When Artifactory is running, run the <code>tools/deploy-infra.sh</code> script from this repo.
The following listing shows the script:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">git clone https://github.com/CloudPipelines/concourse
cd concourse
./tools/deploy-infra.sh</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>As a result, both <code>eureka</code> and <code>stub runner</code> repos are cloned, built,
and uploaded to Artifactory.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="concourse-pcfdev-cf"><a class="link" href="#concourse-pcfdev-cf">Start PCF Dev</a></h4>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title="Tip"></i>
</td>
<td class="content">
You can skip this step if you have CF installed and do not want to use PCF Dev.
The only thing you have to do is to set up spaces.
</td>
</tr>
</table>
</div>
<div class="admonitionblock warning">
<table>
<tr>
<td class="icon">
<i class="fa icon-warning" title="Warning"></i>
</td>
<td class="content">
Servers often run run out of resources at the stage step.
If that happens, <a href="#resources">clear some apps from PCF Dev and continue</a>.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>You have to download and start PCF Dev, as described  <a href="https://pivotal.io/platform/pcf-tutorials/getting-started-with-pivotal-cloud-foundry-dev/install-pcf-dev">here.</a></p>
</div>
<div class="paragraph">
<p>The default credentials for PCF Dev are as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">username: user
password: pass
email: user
org: pcfdev-org
space: pcfdev-space
api: api.local.pcfdev.io</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>You can start the PCF Dev as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">cf dev start</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>You must create three separate spaces, as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">cf login -a https://api.local.pcfdev.io --skip-ssl-validation -u admin -p admin -o pcfdev-org

cf create-space pcfdev-test
cf set-space-role user pcfdev-org pcfdev-test SpaceDeveloper
cf create-space pcfdev-stage
cf set-space-role user pcfdev-org pcfdev-stage SpaceDeveloper
cf create-space pcfdev-prod
cf set-space-role user pcfdev-org pcfdev-prod SpaceDeveloper</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>You can also run the <code>./tools/cf-helper.sh setup-spaces</code> script to create the spaces.</p>
</div>
</div>
<div class="sect3">
<h4 id="concourse-fly-cf"><a class="link" href="#concourse-fly-cf">Setup the <code>fly</code> CLI</a></h4>
<div class="paragraph">
<p>If you go to the Concourse website, you should see something resembling the following image:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/running_concourse.png" alt="running concourse">
</div>
</div>
<div class="paragraph">
<p>You can click one of the icons (depending on your OS) to download <code>fly</code>, which is the Concourse CLI. Once you download that (and maybe, depending on your OS, add it to your PATH) you can run the following command:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">fly --version</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>If <code>fly</code> is properly installed, it should print out the version.</p>
</div>
</div>
<div class="sect3">
<h4 id="concourse-credentials-cf"><a class="link" href="#concourse-credentials-cf">Set up Your <code>credentials.yml</code> File</a></h4>
<div class="paragraph">
<p>The repository comes with <code>credentials-sample-cf.yml</code>, which is set up with sample data (mostly credentials) that are applicable for PCF Dev. Copy this file to a new file called <code>credentials.yml</code> (the file is added to <code>.gitignore</code> so that you cannot push it with your passwords) and edit it as you wish. For our demo, set up the following:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>app-url</code>: URL pointing to your forked <code>github-webhook</code> repository.</p>
</li>
<li>
<p><code>github-private-key</code>: Your private key to clone and tag GitHub repositorys.</p>
</li>
<li>
<p><code>repo-with-binaries</code>: The IP is set to the defaults for Docker Machine. You should update it to point to your setup.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>If you do not have a Docker Machine, run th <code>./whats_my_ip.sh</code> script to
get an external IP that you can pass to your <code>repo-with-binaries</code>, instead of the default
Docker Machine IP.</p>
</div>
<div class="paragraph">
<p>The following table describes the environment variables required by the scripts:</p>
</div>
<table class="tableblock frame-topbot grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property Name</th>
<th class="tableblock halign-left valign-top">Property Description</th>
<th class="tableblock halign-left valign-top">Default value</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_TEST_API_URL</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The URL to the CF Api for TEST env</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>api.local.pcfdev.io</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_STAGE_API_URL</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The URL to the CF Api for STAGE env</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>api.local.pcfdev.io</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_PROD_API_URL</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The URL to the CF Api for PROD env</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>api.local.pcfdev.io</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_TEST_ORG</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Name of the org for the test env</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pcfdev-org</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_TEST_SPACE_PREFIX</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Prefix of the name of the CF space for the test env to which the app name will be appended</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>cloud-pipelines-test</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_STAGE_ORG</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Name of the org for the stage env</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pcfdev-org</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_STAGE_SPACE</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Name of the space for the stage env</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>cloud-pipelines-stage</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_PROD_ORG</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Name of the org for the prod env</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>pcfdev-org</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_PROD_SPACE</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Name of the space for the prod env</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>cloud-pipelines-prod</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>REPO_WITH_BINARIES_FOR_UPLOAD</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">URL to repo with the deployed jars</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code><a href="https://192.168.99.100:8081/artifactory/libs-release-local" class="bare">https://192.168.99.100:8081/artifactory/libs-release-local</a></code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>M2_SETTINGS_REPO_ID</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The id of server from Maven settings.xml</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>artifactory-local</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>PAAS_HOSTNAME_UUID</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Additional suffix for the route. In a shared environment the default routes can be already taken</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
</tbody>
<tfoot>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>BUILD_OPTIONS</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Additional options you would like to pass to the Maven / Gradle build</p></td>
<td class="tableblock halign-left valign-top"></td>
</tr>
</tfoot>
</table>
<div class="paragraph">
<p>The right column shows the default values for PCF Dev that we set in the <code>credentials-sample-cf.yml</code>. <code>PAAS_HOSTNAME_UUID</code> and <code>BUILD_OPTIONS</code> have no default values.</p>
</div>
</div>
<div class="sect3">
<h4 id="concourse-build-cf"><a class="link" href="#concourse-build-cf">Build the Pipeline</a></h4>
<div class="paragraph">
<p>Log in (for example, for a Concourse instance running at <code>192.168.99.100</code>&#8201;&#8212;&#8201;if you do not provide any value, <code>localhost</code> is assumed). If you run the login script, it assumes that either <code>fly</code> is on your <code>PATH</code> or it is in the same folder as the script. The following example shows how to specify an IP address for the login script:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">./login.sh 192.168.99.100</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Next, run the command to create the pipeline, as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">./set_pipeline.sh</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Then you can create a <code>github-webhook</code> pipeline under the <code>docker</code> alias, using the provided <code>credentials.yml</code> file.
You can override these values in exactly that order (for example <code>./set-pipeline.sh some-project another-target some-other-credentials.yml</code>)</p>
</div>
</div>
<div class="sect3">
<h4 id="concourse-run-cf"><a class="link" href="#concourse-run-cf">Run the <code>github-webhook</code> Pipeline</a></h4>
<div class="paragraph">
<p>The following images show the various steps involved in running the <code>github-webhook</code> pipeline:</p>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_login.png" alt="concourse login">
</div>
<div class="title">Step 1: Click <code>Login</code></div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_team_main.png" alt="concourse team main">
</div>
<div class="title">Step 2: Pick <code>main</code> team</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_user_pass.png" alt="concourse user pass">
</div>
<div class="title">Step 3: Log in with <code>concourse</code> user and <code>changeme</code> password</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_pipeline.png" alt="concourse pipeline">
</div>
<div class="title">Step 4: Your screen should look more or less like this</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/start_pipeline.png" alt="start pipeline">
</div>
<div class="title">Step 5: Unpause the pipeline by clicking in the top lefr corner and then clicking the <code>play</code> button</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/generate_version.png" alt="generate version">
</div>
<div class="title">Step 6: Click 'generate-version'</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/run_pipeline.png" alt="run pipeline">
</div>
<div class="title">Step 7: Click <code>+</code> sign to start a new build</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_pending.png" alt="concourse pending">
</div>
<div class="title">Step 8: The job is pending</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/job_running.png" alt="job running">
</div>
<div class="title">Step 9: Job is pending in the main screen</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/running_pipeline.png" alt="running pipeline">
</div>
<div class="title">Step 10: Job is running in the main screen</div>
</div>
</div>
</div>
</div>
</div>
</div>
<link rel="stylesheet" href="js/highlight/styles/atom-one-dark-reasonable.min.css">
<script src="js/highlight/highlight.min.js"></script>
<script>hljs.initHighlighting()</script>
</body>
</html>